Skip to content

HDDS-14578. Ozone admin command gives inconsistent error messages on expired keytab.#10160

Draft
sravani-revuri wants to merge 3 commits intoapache:masterfrom
sravani-revuri:HDDS-14578
Draft

HDDS-14578. Ozone admin command gives inconsistent error messages on expired keytab.#10160
sravani-revuri wants to merge 3 commits intoapache:masterfrom
sravani-revuri:HDDS-14578

Conversation

@sravani-revuri
Copy link
Copy Markdown
Contributor

@sravani-revuri sravani-revuri commented Apr 30, 2026 •
edited
Loading

What changes were proposed in this pull request?

Ozone admin command gives inconsistent error messages on expired keytab for the following commands.

  1. ozone admin safemode status
  2. ozone admin container create
  3. ozone admin container info
bash-5.1$ ozone admin safemode status
Service ID: scmservice
Could not determine leader node

bash-5.1$ ozone admin container create
java.security.cert.CertificateException: org.apache.hadoop.security.AccessControlException: Permission denied.

~# ozone admin container info 69017
Unable to retrieve the container details for 69017
~# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: scm@example.COMValid starting       Expires              Service principal
01/30/2026 10:33:22  01/30/2026 20:33:22  krbtgt/example.COM@example.COM
        renew until 01/31/2026 10:33:10
~# date
Fri Feb  6 12:05:53 PM UTC 2026
root@ve1328:~# ozone admin scm roles
DestHost:destPort ve.example.com:9860 , LocalHost:localPort ve.example.com/10.xx:0. Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[KERBEROS]
root@ve1328:~# ozone admin container info 69017
Unable to retrieve the container details for 69017
~# kinit -kt scm.keytab scm
~# ozone admin container info 69017
Container id: 69017
Pipeline id: 3c8208ce-46ca-4435-8554-f07a58ec6bf7
Write PipelineId: 7a571825-4751-4876-a6ac-acc4c097f282
Write Pipeline State: CLOSED
Container State: CLOSED

Works again on renewing the Kerberos credentials

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-14578

How was this patch tested?

Manual test.

container info command:

bash-5.1$ ozone admin container info 1
Unable to retrieve the container details for 1. DestHost:destPort scm1.org:9860 , LocalHost:localPort scm1.org/172.25.0.116:0. Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[KERBEROS]

container create command:

bash-5.1$ ozone admin container create
java.security.cert.CertificateException: java.io.IOException: Client cannot authenticate via:[KERBEROS]

safemode status command:

bash-5.1$ ozone admin safemode status
Service ID: scmservice
Could not determine leader node. DestHost:destPort scm1.org:9860 , LocalHost:localPort scm1.org/172.25.0.116:0. Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[KERBEROS]

normal execution:

bash-5.1$ kinit -kt /etc/security/keytabs/scm.keytab scm/scm@EXAMPLE.COM

bash-5.1$ ozone admin container create
Container 1 is created with replication config STANDALONE/ONE.

bash-5.1$ ozone admin safemode status
Service ID: scmservice
scm1.org:9860 [scm1]: out of safe mode

bash-5.1$ ozone admin container info 1
Container id: 1
Pipeline id: 9be87dfe-be2f-409f-8650-9b9ad05115ba
Write PipelineId: 9be87dfe-be2f-409f-8650-9b9ad05115ba
Write Pipeline State: OPEN
Container State: OPEN
SequenceId: 0
Datanodes: [c15fd59a-af5e-411f-8118-e5014ca3584e/ozonesecure-ha-datanode3-1.ozonesecure-ha_ozone_net]
Replicas: [State: OPEN; ReplicaIndex: 0; SequenceId: 0; Origin: c15fd59a-af5e-411f-8118-e5014ca3584e; Location: c15fd59a-af5e-411f-8118-e5014ca3584e/ozonesecure-ha-datanode3-1.ozonesecure-ha_ozone_net]

@Gargi-jais11 Gargi-jais11 self-requested a review April 30, 2026 04:21
@sravani-revuri sravani-revuri changed the title HDDS-14578. Ozone admin command gives Unable to retrieve the container details on expired keytab HDDS-14578. Ozone admin command gives inconsistent error messages on expired keytab. Apr 30, 2026
@adoroszlai
Copy link
Copy Markdown
Contributor

adoroszlai commented Apr 30, 2026

Thanks @sravani-revuri for the patch. Are you planning to make admin commands more consistent, or just clarifying admin container info with the exception message?

@sravani-revuri
Copy link
Copy Markdown
Contributor Author

sravani-revuri commented Apr 30, 2026 •
edited
Loading

Thanks @sravani-revuri for the patch. Are you planning to make admin commands more consistent, or just clarifying admin container info with the exception message?

I will be doing it for the other admin commands error messages mentioned in the jira. the jira was initially just for the container info command, but i had found some more inconsistencies. Updated the jira to handle it in this PR itself.

Will update the testing proof once i'm done.

@Gargi-jais11
Copy link
Copy Markdown
Contributor

Gargi-jais11 commented Apr 30, 2026

@sravani-revuri For the other ozone admin commands which need to have clear error message, please update about it in the PR description.

@sravani-revuri
Copy link
Copy Markdown
Contributor Author

sravani-revuri commented Apr 30, 2026

@Gargi-jais11 could you please take a look at these changes?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Gargi-jais11 Gargi-jais11 Awaiting requested review from Gargi-jais11

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@sravani-revuri @adoroszlai @Gargi-jais11